Papers with regularization technique
Improving Low-Resource Cross-lingual Parsing with Expected Statistic Regularization (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing methods for cross-lingual syntactic analysis have been shown to be effective for low-resource languages. |
| Approach: | They propose to use low-order statistical functions to shape model distributions for semi-supervised learning on low-resource datasets. |
| Outcome: | The proposed method improves POS and LAS on 5 target languages and provides significant gains over strong cross-lingual-transfer-plus-fine-tuning baselines for modest amounts of label data. |
Effective Adversarial Regularization for Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Existing (small) perturbations that induce a critical prediction error in machine learning models are often referred to as adversarial examples. |
| Approach: | They propose to use adversarial perturbations to regularize text classification tasks by adding adversarials to a typical NMT model structure. |
| Outcome: | The proposed method significantly improves performance of NMT models, such as LSTM-based and Transformer-based models. |
CharSpan: Utilizing Lexical Similarity to Enable Zero-Shot Machine Translation for Extremely Low-resource Languages (2024.eacl-short)
Copied to clipboard
| Challenge: | Existing models for ELRLs lack parallel corpora and monolingual corporata . authors propose novel character-span noise argumentation model to facilitate cross-lingual transfer . |
| Approach: | They propose a character-span noise argumentation model to facilitate cross-lingual transfer . they use character-size noise argumentations to regularize training data of HRL . |
| Outcome: | The proposed model outperforms baselines on closely related HRL-ELRL pairs from three different language families. |
Beyond Reptile: Meta-Learned Dot-Product Maximization between Gradients for Improved Single-Task Regularization (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to improve generalization of neural models use a small component of the gradient for maximizing dot-product between batches. |
| Approach: | They propose to use a finite differences first-order algorithm to calculate a gradient from dot-product of gradients and regularize it. |
| Outcome: | The proposed method outperforms previous approaches of Reptile and MAML when used as a regularization technique. |
ReWE: Regressing Word Embeddings for Regularization of Neural Machine Translation Systems (N19-1)
Copied to clipboard
| Challenge: | Existing methods to regularize neural machine translation are limited in low-resource settings. |
| Approach: | They propose a method that uses regressing word embeddings to regularize neural machine translation. |
| Outcome: | The proposed system improves on a strong baseline and a state-of-the-art system. |
Document-Level Event Argument Extraction via Optimal Transport (2022.findings-acl)
Copied to clipboard
| Challenge: | Prior work on event-level EAE models ignore syntactic structures for documents . prior work on EE is restricted to sentence-level setting where event triggers and arguments are assumed to appear in the same sentences. |
| Approach: | They propose to employ Optimal Transport to induce structures of documents based on sentence-level syntactic structures and tailored to EAE task. |
| Outcome: | The proposed model is effective in document-level EAE, with a new constraint on unrelated context words. |
Smoothing and Shrinking the Sparse Seq2Seq Search Space (2021.naacl-main)
Copied to clipboard
| Challenge: | entmax-based sparse sequence-to-sequence models give high scores to short hypotheses . ent max models can shrink the search space by assigning zero probability to bad hypothese . |
| Approach: | They propose entmax-based sparse sequence-to-sequence models that minimize cross-entropy and use softmax to compute local normalized probabilities over target sequences. |
| Outcome: | The proposed models remove a major source of model error for word-level tasks . the proposed models improve cross-lingual morphological inflection and machine translation . |
LinguAlchemy: Fusing Typological and Geographical Elements for Unseen Language Generalization (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained language models have shown remarkable generalization toward multiple tasks and languages, but their generalization towards unseen languages is poor. |
| Approach: | They propose a regularization technique that incorporates various aspects of languages to better characterize linguistics constraints. |
| Outcome: | The proposed technique improves accuracy of mBERT and XLM-R on unseen languages by 18% and 2% compared to fully finetuned models. |
Domain Adversarial Fine-Tuning as an Effective Regularizer (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing fine-tuning techniques can degrade general-domain representations . however, fine-timing can lead to catastrophic forgetting of knowledge . |
| Approach: | They propose a new regularization technique that complements the task-specific loss used during fine-tuning with an adversarial objective. |
| Outcome: | Empirical results show that AFTER improves performance on various natural language understanding tasks compared to standard fine-tuning. |
The Role of n-gram Smoothing in the Age of Neural Networks (2024.naacl-long)
Copied to clipboard
| Challenge: | n-gram smoothing techniques were used to overcome overfitting problems in neural language models for decades. |
| Approach: | They propose to convert any n-gram smoothing technique into a regularizer compatible with neural language models. |
| Outcome: | The proposed regularizers outperform label smoothing on language modeling and machine translation. |
Contextualizing Hate Speech Classifiers with Post-hoc Explanation (2020.acl-main)
Copied to clipboard
| Challenge: | Modern text classifiers struggle to learn a model of hate speech that generalizes to real-world applications. |
| Approach: | They propose a method to regularize BERT classifiers to detect bias towards identity terms by providing explanations for group identifiers and allowing models to learn from the context of group identifiers. |
| Outcome: | The proposed method limiting false positives on out-of-domain data while maintaining and improving in-domain performance. |
Boundary Smoothing for Named Entity Recognition (2022.acl-long)
Copied to clipboard
| Challenge: | Named entity recognition models often encounter over-confidence issues . boundary smoothing is a method that re-assigns entity probabilities from annotated spans to the surrounding ones . |
| Approach: | They propose a method for regularizing entity probabilities from annotated spans to the surrounding ones. |
| Outcome: | The proposed method achieves better than or competitive with previous state-of-the-art systems on well-known benchmarks. |
Bridging Robustness and Generalization Against Word Substitution Attacks in NLP via the Growth Bound Matrix Approach (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have shown that adversarial examples can alter models' predicted sentiment due to their sensitivity to specific word choices. |
| Approach: | They propose a regularization technique to improve NLP model robustness by reducing the impact of input perturbations on model outputs. |
| Outcome: | The proposed method outperforms state-of-the-art methods in adversarial defense. |
Introducing Syntactic Structures into Target Opinion Word Extraction with Deep Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Current deep learning models fail to exploit syntactic information of sentences . proposed model incorporates syntax-based opinion possibility scores and syntaktic connections between the words . |
| Approach: | They propose to incorporate syntactic information of sentences into deep learning models for TOWE . they propose a novel regularization technique to improve the performance of the models . |
| Outcome: | The proposed model achieves state-of-the-art on four benchmark datasets. |